Description

I found that this test is flakily fails with the below assertion failure. Hence, I suggest a new way to fix the test by adding some synchronization for the test execution only. I at first identify the source code location whose slow execution leads to the flaky test failure, where if org/java_websocket/WebSocketImpl.java#572 slows to set the value of readyState then the test fails (org.java_websocket.issues.Issue677Test#119). Hence, I introduce one variable in this class [WebSocketImpl.java] that is only there to provide some synchronization. By analyzing the code, I found if the assertion check happens before the 2times execution of this line, then the assertion is failing. Hence, until this statement is executed two times, I force the thread that the test runs on to wait before it accesses the value of webSocket0.isClosed(). The waiting location is at before org.java_websocket.issues.Issue677Test#119.

Related Issue

Running org.java_websocket.issues.Issue677Test
Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.286 sec <<< FAILURE!
testIssue(org.java_websocket.issues.Issue677Test) Time elapsed: 0.265 sec <<< FAILURE!
java.lang.AssertionError: webSocket.isClosed()
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.assertTrue(Assert.java:41)
at org.java_websocket.issues.Issue677Test.testIssue(Issue677Test.java:119)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:242)
at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:137)
at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)

Results :

Failed tests: testIssue(org.java_websocket.issues.Issue677Test): webSocket.isClosed()

Tests run: 1, Failures: 1, Errors: 0, Skipped: 0

Motivation and Context

The test is flaky, and it is passing and failing in different runs.

How Has This Been Tested?

I run this test 1000 times on Ubuntu 20.04 with Java 11 and it always passes.

Types of changes

• Flaky Test fix (non-breaking change which fixes an issue)

Checklist:

• My code follows the code style of this project.
• All new and existing tests passed.